3D imaging techniques have been demonstrated ranging from con-focal microscopes to time-of-flight laser rangefinders, covering applications from microscopic integrated circuit measurements to large field-of-view applications like mobile robot navigation. Performance specifications for these sensors vary widely, with a figure of merit often determined based upon specific requirements.
Many applications impose simultaneous requirements of rapid inspection rates for 100% inspection of the entire object volume, yet accuracy and precision measured in microns or perhaps finer. Such requirements present challenges for sensor design, and necessary consideration of fundamental imaging limits. These limits require not only examination of the technical specifications of scanning and detection devices used for image formation, but close examination of the characteristics (reflection, transmission, size relative to sensor resolution) of the objects being examined. In a 3D system, the reproducibility of a measurement is often limited by sensor parameters (signal to noise, resolution, etc.) but the absolute accuracy is limited by the object structure and the influence of optical background (both spatial and temporal noise.
With increasing trends toward miniaturization, manufacturers of microelectronics assemblies and miniature parts often have requirements for very fine measurement or defect detection capability for automatic three-dimensional optical inspection equipment. 3D sensors which can potentially satisfy the measurement requirements include confocal microscopes, dynamic focus sensors, triangulation-based probes, and various interferometric devices. Cost effective solutions for process monitoring where specific regions of interest are monitored now exist, but are often limited in measurement speed.
For example, the requirements for 100% inspection of a 641 .times.641 wafer (or other object) with a 2.5 .mu.m.times.2.5 .mu.m (which can be coarse for inspection of such devices) samples taken over the entire surface are extreme. A typical dynamic focus probe acquires about 1000 data points per second. The resulting inspection time for a 641 .times.641 object is 1.3 months. Point triangulation sensors operate at about 10000 points per second and reduce the inspection time to a few days. 3D sensors which approach video rates, as described in U.S. Pat. No. 5,024,529 reduce the inspection time to minutes which is acceptable for many facilities.
Also, other applications for in-line 100% inspection may require a few seconds for 100% inspection of a 641 .times.641 part with coarser resolution (say 25 .mu.m.times.25 .mu.m). In many applications, a tradeoff between speed and accuracy is not acceptable which restricts many present measurement tools to off-line inspection which greatly reduces the value added. Hence, accuracy and high speed, which are mutually conflicting parameters, are both required to maximize the return on investment of inspection equipment. Clearly, very rapid data rates must be achieved for 100% inspection.
A three dimensional imager which can meet accuracy requirements and speed requirements has a recognized need in at least the following areas:
Inspection of circuit boards and components: solder paste and components, lead and solder ball grid array coplanarity, package dimensions and lead to body standoff; PA1 Microelectronics assemblies: multi-chip-modules, high density miniature bump grid arrays, TAB inner lead bond, wire bond; PA1 Micromachined parts: miniature machines and mechanical assemblies; PA1 Data storage industry: disk platter flatness, disk drive suspensions, defect detection; PA1 General surface defect detection: ceramics, glass, metals, paper; and PA1 Shape analysis and matching: forensic science including toolmarks, firearm ID, pattern matching, 3D texture analysis.
Of all known methods for 3D imaging, triangulation provides the most practical method for a high speed-accuracy product. Triangulation is the most effective method for applications where a relatively short working distance is acceptable and where off-axis viewing of three dimensional shape is tolerable (i.e. no narrow, deep holes).
FIG. 1 shows a sketch of a typical triangulation based sensor. An incident laser beam 10 strikes an object 12 at a first or a second height 14 and 16, respectively. A position sensitive detector or array sensor 18 is provided as well as a receiver optical system 20 for magnification. The instantaneous field of view of the sensor 18 is indicated at 22. Height and gray scale information are provided on lines 24 from a signal processor 26 to a computer 28 for image processing and analysis.
Although the triangulation method provides many advantages, it is not immune to certain types of errors, the most obvious of which are due to shadowing or occlusion as shown in FIG. 2. These problems are compensated for by the use of two or more sensors 30 and 31. An incident laser beam 32 strikes a step object 34 which has an occluded region 36 wherein sensor rays from the surface of the object are blocked from the sensor 30. However, this region 36 is visible to the sensor 31 as indicated at line 38. The IFOV of both sensors 30 and 31 is indicated at 39.
More detailed examination of potential defects shows other important errors arise from acceptance of scattered light and various sources of optical/electronic background noise within the instantaneous field of view (IFOV). In triangulation systems, unlike dynamic focus sensors, the IFOV along the triangulation sensing axis cannot be reduced to a single point, and the system designer must be prepared to deal with this limitation.
It is important to recognize that simple occlusion is a relatively easy problem to solve with just a pair of detectors, as shown in FIG. 2. Often a sum or average of recorded height values is used for an approximation, and if the shadow is perfect (no stray light leakage), the proper value will be recorded. Also, error contributions created by non-symmetric spot distributions resulting from surface roughness/marks in non-occluded or quasi-flat areas are significantly reduced by averaging the asymmetric spot distribution seen along the position sensing axis. This is a consequence of imaging from opposite viewpoints where if the peak of the asymmetric spot distribution appears high to detector #1, it will appear low to detector #2. The average (or midpoint for two detectors) is a good estimate of the spot center, and the error cancels or is greatly reduced.
However, examination of complex surfaces indicates that sensor processing techniques which only add or average multiple channels do not eliminate or even identify some of the most severe errors. In FIG. 3, an incident laser beam 40 penetrates translucent material of an object 42 and propagates outward becoming visible as background light to a first sensor 44 wherein the object 42 blocks light from a scattering center invisible to the first sensor 44. At a region 45 of the object 42, scattered light is visible to both detectors. The light is scattered from internal scattering centers 43. The average value no longer is a proper height measurement but is erroneously recorded as a very high peak when, in fact, the region is much lower. True surface position is indicated at 46 and a truncated (shadowed) spot distribution is indicated at 48. At a second sensor 50, the spot distribution is shown at 52. At region 47, scattered light is only visible to the second sensor 50. A ray from the scattering center seen by the sensor 50 is indicated at 49.
This severe error illustrated in FIG. 3 limits proper discrimination of the object and background on the basis of height, and even capability for "presence/absence" detection.
Similarly, FIG. 4 shows a case often found on circuit boards where a series of pads 54 are covered with solder paste or, on a finer scale, where leads are connected to the bonding pads in the inner lead bond region of a tab integrated circuit. FIG. 4 shows first and second laser light scans 51 and 53 and their associated first and second detectors 56 and 58, respectively. A line 60 indicates current detector position and spots on the detectors 56 and 58 represent actual detector position.
The pads 54 are in close proximity, and the slanted edges lead to multiple reflections from adjacent pads or secondary background illumination as indicated at line 62. Lines 64 indicate useful distance reflection from the point of interest. The sensor sees a secondary reflection from adjacent pad resulting in an erroneous high value, but the noise is occluded by the sensor 58, so the error cannot cancel with simple averaging. Hence, there is an artifact at the edge manifested as an incorrect high Z value.
Another important example is shown in FIG. 5. The object has a shiny surface finish and a curved geometric shape, the configuration which is most challenging for any 3D imaging system. A specific example is a single solder ball 66 which is an element of a ball grid array (BGA) consisting of hundreds or even thousands of solder locations per square inch. Dimensions of each solder ball can vary from about ten microns to about 0.5 mm. With a high density of small diameter features, inspection will only prove useful if the imaging and image analysis system are able to provide measurements indicating deviation of coplanarity (height) and ball diameter measurements at a very rapid rate of speed, perhaps several thousand per second. Examination of three representative scans across the object illustrate challenges in imaging and preprocessing.
Scan #1 (not shown in FIG. 5). When the scanning laser beam is near the peak of the solder ball, the gray scale and height values will be well matched and well within the dynamic range of the sensor, and the average height value is a good estimate and cancels some asymmetry in the spot distribution. The surface orientation is favorable for both sensors 68 and 70.
Scan #2. However, for a second scan for an incident laser beam 72 where the surface normal is such that sensor 68 is receiving direct specular reflection, a very high gray scale value will be recorded at the sensor 68. This signal can easily exceed the dynamic range of the system, even if high speed automatic light control (ALC) is used. If ALC is used to bring the sensor 68 into range (say, by reducing signal by one to two decades), sensor 70 may receive low light return resulting in a poor signal to noise ratio and an inaccurate estimate of the height. Although the sensor is brought into the intensity range, the local dynamic range is so large that optical crosstalk (smear and streak noise) may result in corruption of the height values in the region near the saturation point because signal may vary by about four decades in a local region.
Furthermore, if the ALC range is extreme, measurement speed may be compromised. If the range of ALC is insufficient and compression of the strong signal does not produce valid data for the sensor 68, then the sensor 70 will also be out of range. If direct laser diode modulation (only one decade range) is incorporated, this situation can easily occur. Hence, the strongest signal may not always be best.
Scan #3. A third scan down the side of the object shows a most difficult problem. The specular surface illuminates what may be a glossy background which, because of the angle of incidence and reflection, produces an intensity reading at the sensor 68 which is, perhaps, orders of magnitude stronger than the signal from the point of interest. In this case, both the height and intensity readings from the sensors 68 and 70 are much different and indicate a severe error condition and low confidence in the data.
These illustrations show that for scanning of a single solder ball, each sensor is required to operate throughout the entire dynamic range, and will encounter spurious data at many points. For measurement of grids, the luxury of re-orienting the sensor through rotation of articulation cannot be afforded if 100% inspection is to be accomplished at assembly line rates. Furthermore, for such a geometry, no favorable position for the sensor may exist which eliminates the error. Nevertheless, key information for the measurement task is still available, provided erroneous data points can be recognized or filtered. Many other examples can be found where simple averaging of height values (suggested in prior art) produces large measurement errors, particularly for complex shapes with smooth surface finish.
Errors associated with these phenomena have been quantified. A typical problematic case arises when inspecting solder paste and traces on printed circuit boards. The effect of occlusion and reflection artifacts is best illustrated by comparing results for long, thin objects which are measured, then rotated by 90 degrees, and measured again. Test results indicated variation in readings from about 30% to 50% depending upon the reflectance variation between the solder paste, background, and pads.
The error is noticeable when the occluded region as illustrated in FIG. 4 is along the length of the pad, corresponding to the position sensitive dimension. The artifacts not only are manifested in the computation, but are so large as to be visible to the eye. Such variation is unacceptable for in-line process control, and because of the visibility, barely acceptable for "presence/absence" (coarse inspection).
Many triangulation based probes are available and can be broadly classified as follows: point sensors, line of light systems, and laser scanners producing data rates of about 10K points per second, up to 100K points per second, and up to 10M points per second respectively. Regardless of the particular embodiment, it is well known among those skilled in triangulation based imaging that the use of various combinations of multiple transmitter and receiver devices can be used to improve performance through reduction of shadowing and occlusion related defects as illustrated in FIG. 4. Aside from the disadvantage of slow measurement speed, point sensors have been developed with 4 detector systems and 8 detector systems arranged symmetrically about the transmitted point of light. In 4 detector systems, the receivers are positioned at the sides square, and in a ring detector arrangement for 8 detector systems. There is obvious benefit to this approach, but upon consideration of inspection speed requirements, the method is not acceptable.
In an array camera system, the errors resulting from reflection phenomena shown in FIGS. 3-5 can sometimes be reduced by counting and suppressing multiple spots found along the position sensing axis, but only at the penalty of severely reduced measurement speed. Furthermore, the correct height value cannot be discerned simply by looking at the intensity of each spot, sometimes the erroneous value may have a larger intensity, as shown in FIG. 5.
Line of light systems offer substantial measurement speed advantages over mechanically scanned point detectors, but are limited by the camera readout rate to the 30K point per second range. Multiple lines of light, high speed framing cameras, or smaller arrays which can be read at faster frame rates offer some advantages. Nevertheless, accuracy will often be unacceptable in measurement applications (as opposed to height presence/absence) unless scene dependent errors are reduced. In laser line of light or line scan systems, the symmetry which allows for simple averaging of non-occluded detector pairs is absent, restricting the number of detectors to 2 devices, unless successive scans are utilized with rotation of the imaging head or part, or additional cameras and scan lines added for orthogonal viewing.
The U.S. Pat. No. to Clerget, 4,534,650, describes a 3D measurement device utilizing at least two distinct sensors (photodiode bars). Part of the purpose of the part is to correct output from "faulty photosensitive elements". As can be seen from the above discussion of reflections and occlusion, that is the least of the problems which can occur.
The U.S. Pat. Nos. to Case et al., 4,891,772 and 4,733,969, describe the use of a "plurality" of sensors for line and point measurements. Features include exposure control based upon spot size measurement so as to allow for use of nxn array or sensors and "low level" algorithms for estimation of the spot center from the intensity distribution obtained from readout of each CCD array. In the '969 patent it was stated in column 2 that a "user of the system may reject certain areas and eliminate erroneous readings caused by multiple reflections from complex geometries". This allows the user access to image for placing "windows".
In the '772 patent (also using multiple detecting elements), the exposure is controlled to balance power on detector.
The U.S. Pat. No. to Stern, 4,643,578, describes a system that uses a mask synchronized with the position of a light beam and readout location. This method is very effective for reduction of errors from stray light outside the FOV, and to mask high background optical noise as might be encountered in a welding application. However, the entire position sensing area cannot be masked effectively so that severe errors can still be present.
The U.S. Pat. Nos. to Penney et al., 4,634,879 and 4,645,917, describe a means of filtering optical noise similar to Stern, but the system employs photo-multiplier tubes for position detection rather than a CCD array. The descanning action performs the same function as shown by Stern. Only a single receiver is used and occlusion effects limit performance.
The U.S. Pat. No. to Nakawaga, 4,553,844, shows a scanning/descanning arrangement for inspecting solder. The direction orthogonal to the position sensing axis is blocked resulting in an IFOV which is a narrow strip, similar to the Penney and Stern disclosures. The position sensing axis is exposed and with the use of only a single detection device the system is limited by phenomena illustrated in FIGS. 4 and 5. The most severe errors are not compensated. Light which is blocked often has lower intensity than "highlights" found along PSD axis.
The U.S. Pat. No. to Amir, 5,048,965, describes a 3D imaging method using a CCD linear array and matched pairs of lights for occlusion avoidance. The method has potential for high speed acquisition (approaching video rates) and, with a 4096 linear array, simultaneous wide field coverage. With this method, two images, each with two matched light sources, are required for occlusion avoidance. Instead of using a position sensitive detector (PSD) for each receiver, the equivalent function is provided by a pair of encoded light patterns (illumination gradients of opposite sign) which are projected onto the surface and imaged onto the array, one illumination pattern for each single scan of the object. The height measurement obtained from a single illumination direction is computed with ratiometric processing (similar to PSD based systems) applied to the two encoded patterns (which are switched between passes). For occlusion avoidance, a prescribed relationship between the intensity values is used to compute depth as desired from the illumination spatial profiles. Examination of the resulting equation shows the relationship is equivalent to averaging the centroid estimates obtained from the two illumination directions.
The U.S. Pat. No. to Chen et al., 5,118,192, shows a laser based 3D sensor system with internal mechanical means of rotating sensors to achieve favorable viewing conditions of irregular objects. The system utilizes multiple laser transmitters and detectors. The arrangement is optimized for viewing of objects like solder joints where occlusion and reflection artifacts are similar as illustrated in FIG. 5. The oblique illumination and viewing arrangement, together with a means of pivoting the sensor, allow a substantial fraction of the sloped sides of the solder joint to be viewed in such a manner that a high signal to noise--low spurious reflection condition--is achieved for typical surface orientation and conditions. However, several conditions can be found by ray tracing for which occlusion and reflection artifacts will be present within any single image. For demanding high speed applications, the time to acquire and process multiple viewpoints acquired through rotations and articulation is likely not available.
Other systems utilizing multiple triangulation-based detectors are disclosed in the U.S. Pat. No. to Hata, 4,731,853, which uses CCD detectors, and in the article by Kooijman, K., Horijon, J., (Philips) "Video Rate Laser Scanner: Considerations on Triangulation, Optics, Detectors, and Processing Circuits", Proc. SPIE, Vol. 2065, D. Svetkoff ed., September 1993, p. 253, Sec. 2, Triangulation Optics.